快速移动受试者的运动模糊是摄影中的一个长期问题,由于收集效率有限,尤其是在弱光条件下,在手机上非常常见。尽管近年来我们目睹了图像脱毛的巨大进展,但大多数方法都需要显着的计算能力,并且在处理高分辨率照片的情况下具有严重的局部动作。为此,我们根据手机的双摄像头融合技术开发了一种新颖的面部脱毛系统。该系统检测到主题运动以动态启用参考摄像头,例如,最近在高级手机上通常可用的Ultrawide Angle摄像机,并捕获带有更快快门设置的辅助照片。虽然主镜头是低噪音但模糊的,但参考镜头却很锋利,但嘈杂。我们学习ML模型,以对齐和融合这两张镜头,并在没有运动模糊的情况下输出清晰的照片。我们的算法在Google Pixel 6上有效运行,每次拍摄需要463毫秒的开销。我们的实验证明了系统对替代单片,多帧,面部特异性和视频脱张算法以及商业产品的优势和鲁棒性。据我们所知,我们的工作是第一个用于面部运动脱毛的移动解决方案,在各种运动和照明条件下,在数千个图像中可靠地工作。
translated by 谷歌翻译
视频博客和自拍照是流行的社交媒体格式,通常由广角相机捕获,以显示人类受试者和扩展的背景。遗憾的是,由于透视投影,靠近角落和边缘的面孔表现出明显的扭曲,延伸并挤出面部特征,导致视频质量差。在这项工作中,我们展示了一种视频扭曲算法来纠正这些扭曲。我们的主要思想是在面部地区本地应用立体投影。我们使用空间 - 时间能量最小化配制网眼翘曲问题,并使用线路保存术语最小化背景变形,以维持背景中的直边。为了解决时间一致性,我们通过潜在变量限制了翘曲网格和面部轨迹上的时间平滑度。对于性能评估,我们开发了具有广泛焦距的广角视频数据集。用户学习表明,83.9%的用户更喜欢基于透视投影的其他替代方案的算法。
translated by 谷歌翻译
This paper describes a Relevance-Zone pattern table (RZT) that can be used to replace a traditional transposition table. An RZT stores exact game values for patterns that are discovered during a Relevance-Zone-Based Search (RZS), which is the current state-of-the-art in solving L&D problems in Go. Positions that share the same pattern can reuse the same exact game value in the RZT. The pattern matching scheme for RZTs is implemented using a radix tree, taking into consideration patterns with different shapes. To improve the efficiency of table lookups, we designed a heuristic that prevents redundant lookups. The heuristic can safely skip previously queried patterns for a given position, reducing the overhead to 10% of the original cost. We also analyze the time complexity of the RZT both theoretically and empirically. Experiments show the overhead of traversing the radix tree in practice during lookup remain flat logarithmically in relation to the number of entries stored in the table. Experiments also show that the use of an RZT instead of a traditional transposition table significantly reduces the number of searched nodes on two data sets of 7x7 and 19x19 L&D Go problems.
translated by 谷歌翻译
This paper proposes a new method, OFA-OCR, to transfer multimodal pretrained models to text recognition. Specifically, we recast text recognition as image captioning and directly transfer a unified vision-language pretrained model to the end task. Without pretraining on large-scale annotated or synthetic text recognition data, OFA-OCR outperforms the baselines and achieves state-of-the-art performance in the Chinese text recognition benchmark. Additionally, we construct an OCR pipeline with OFA-OCR, and we demonstrate that it can achieve competitive performance with the product-level API. The code (https://github.com/OFA-Sys/OFA) and demo (https://modelscope.cn/studios/damo/ofa_ocr_pipeline/summary) are publicly available.
translated by 谷歌翻译
Generalist models, which are capable of performing diverse multi-modal tasks in a task-agnostic way within a single model, have been explored recently. Being, hopefully, an alternative to approaching general-purpose AI, existing generalist models are still at an early stage, where modality and task coverage is limited. To empower multi-modal task-scaling and speed up this line of research, we release a generalist model learning system, OFASys, built on top of a declarative task interface named multi-modal instruction. At the core of OFASys is the idea of decoupling multi-modal task representations from the underlying model implementations. In OFASys, a task involving multiple modalities can be defined declaratively even with just a single line of code. The system automatically generates task plans from such instructions for training and inference. It also facilitates multi-task training for diverse multi-modal workloads. As a starting point, we provide presets of 7 different modalities and 23 highly-diverse example tasks in OFASys, with which we also develop a first-in-kind, single model, OFA+, that can handle text, image, speech, video, and motion data. The single OFA+ model achieves 95% performance in average with only 16% parameters of 15 task-finetuned models, showcasing the performance reliability of multi-modal task-scaling provided by OFASys. Available at https://github.com/OFA-Sys/OFASys
translated by 谷歌翻译
Parameter-efficient methods (like Prompt or Adapters) for adapting pre-trained language models to downstream tasks have been popular recently. However, hindrances still prevent these methods from reaching their full potential. For example, two significant challenges are few-shot adaptation and cross-task generalization ability. To tackle these issues, we propose a general framework to enhance the few-shot adaptation and cross-domain generalization ability of parameter-efficient methods. In our framework, we prime the self-supervised model for parameter-efficient methods to rapidly adapt to various downstream few-shot tasks. To evaluate the authentic generalization ability of these parameter-efficient methods, we conduct experiments on a few-shot cross-domain benchmark containing 160 diverse NLP tasks. The experiment result reveals that priming by tuning PLM only with extra training tasks leads to the best performance. Also, we perform a comprehensive analysis of various parameter-efficient methods under few-shot cross-domain scenarios.
translated by 谷歌翻译
Choosing which properties of the data to use as input to multivariate decision algorithms -- a.k.a. feature selection -- is an important step in solving any problem with machine learning. While there is a clear trend towards training sophisticated deep networks on large numbers of relatively unprocessed inputs (so-called automated feature engineering), for many tasks in physics, sets of theoretically well-motivated and well-understood features already exist. Working with such features can bring many benefits, including greater interpretability, reduced training and run time, and enhanced stability and robustness. We develop a new feature selection method based on Distance Correlation (DisCo), and demonstrate its effectiveness on the tasks of boosted top- and $W$-tagging. Using our method to select features from a set of over 7,000 energy flow polynomials, we show that we can match the performance of much deeper architectures, by using only ten features and two orders-of-magnitude fewer model parameters.
translated by 谷歌翻译
在制定政策指南时,随机对照试验(RCT)代表了黄金标准。但是,RCT通常是狭窄的,并且缺乏更广泛的感兴趣人群的数据。这些人群中的因果效应通常是使用观察数据集估算的,这可能会遭受未观察到的混杂和选择偏见。考虑到一组观察估计(例如,来自多项研究),我们提出了一个试图拒绝偏见的观察性估计值的元偏值。我们使用验证效应,可以从RCT和观察数据中推断出的因果效应。在拒绝未通过此测试的估计器之后,我们对RCT中未观察到的亚组的外推性效应产生了保守的置信区间。假设至少一个观察估计量在验证和外推效果方面是渐近正常且一致的,我们为我们算法输出的间隔的覆盖率概率提供了保证。为了促进在跨数据集的因果效应运输的设置中,我们给出的条件下,即使使用灵活的机器学习方法用于估计滋扰参数,群体平均治疗效应的双重稳定估计值也是渐近的正常。我们说明了方法在半合成和现实世界数据集上的特性,并表明它与标准的荟萃分析技术相比。
translated by 谷歌翻译
策略培训是一种多学科的康复方法,它教导技能减少中风后认知障碍者的残疾。与传统的康复方法相比,在随机,对照临床试验中已显示策略培训是促进独立性的更可行和有效的干预措施。标准化的保真度评估用于通过检查康复视频记录中的指导和定向口头提示来衡量治疗原则的依从性。尽管用于检测指导和定向的口头提示的忠诚度评估对于单一站点研究是有效的,但在大型多站点务实的务实试验中,它可能会变成劳动力密集,耗时且昂贵。为了应对广泛的战略培训实施的这一挑战,我们利用自然语言处理(NLP)技术来自动化策略培训保真度评估,即自动从康复会议的视频记录中自动识别有指导和指导的口头提示。我们开发了一种基于规则的NLP算法,一个长期术语存储器(LSTM)模型以及该任务的变压器(BERT)模型的双向编码器表示。 BERT模型以0.8075的F1得分实现了最佳性能。这项研究的发现在心理学和康复干预研究和实践方面具有广泛的希望。
translated by 谷歌翻译
柔性章鱼臂具有卓越的能力,可以协调大量自由度并执行复杂的操纵任务。结果,这些系统继续吸引生物学家和机器人的注意力。在本文中,我们开发了一个三维模型的软章鱼臂,配备了生物力学上逼真的肌肉致动。考虑了所有主要肌肉群施加的内力和夫妇。描述了一种能量塑形控制方法来协调肌肉活动,以便在3D空间中掌握和触及。本文的主要贡献是:(i)主要肌肉群建模以引起三维运动; (ii)基于存储的能量功能的肌肉激活的数学公式; (iii)通过在特殊欧几里得组SE中解决优化问题获得的设计特定于任务的平衡配置的计算有效过程(3)。然后,根据优化问题解决方案引起的共同状态变量,对肌肉控制进行迭代计算。该方法在物理准确的软件环境弹性中得到了数值的证明。报告了模拟观察到的章鱼行为的数值实验的结果。
translated by 谷歌翻译